Method and system to provide a computer-modified visualization of the desired face of a person

ABSTRACT

A data set of visuals of faces and extracted face property data are generated and linked to face characteristics data provided by a representative set of humans that rate the visuals of these faces with respect to their face characteristics. Further face property data of these visuals of faces is extracted and together with the generated data set used to train an artificial intelligence. The artificial intelligence is used to analyse a visual of the person&#39;s face and generate a data set of modifications based on a selected desired characteristic(s) and modifications achievable by at least one cosmetic and/or medical treatment. The visual of the face of the person is modified based on the data set of modifications and the computer-modified visual of the desired face of the person with the modification of the face achievable by the least one proposed cosmetic and/or medical treatment is generated and displayed.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.17/117,805, filed Dec. 10, 2020, which claims benefit of European PatentApplication Serial No. 19215134.8, filed Dec. 11, 2019, and whichapplications are incorporated herein by reference in their entirety. Tothe extent appropriate, a claim of priority is made to the abovedisclosed applications.

BACKGROUND

The present invention relates to a method and system to provide acomputer-modified visualization of a desired face of a person whoconsiders undergoing a minimally invasive and/or invasive cosmeticand/or medical treatment to improve the person's appearance. There is ageneral wish to optimize the own appearance. The face is one of the mainareas of the body relevant for this appearance. There are many differenttreatments known to change the facial appearance, expression and/ormorphology, e.g. reduction of wrinkles in the skin of the face ormodification of the cheekbones. A person interested in such a changetypically makes an appointment with a beautician, dermatologist,physician, specialist certified to do facial modifications, or plasticsurgeon to get information about possible treatments. In a first stepthe specialist mentioned above inspects the different regions of theface and based on their personal knowledge and skills proposes medicaland/or cosmetic treatments that could provide the desired change of theappearance. The problem of this way of working is that it is difficultfor the person to understand the possible visual effects and/or effectson the first impression (i.e., how the face is perceived by others) ofthe treatments the specialist proposes.

LIAO YANBING ET AL: “Deep Rank Learning for Facial Attractiveness”, 20174^(TH) IAPR ASIAN CONFERENCE ON PATTERN RECOGNITION (ACPR), IEEE, 26Nov. 2017 (2017-11-26), pages 565-570, XP033475316 discloses a DeepConvolutional Neuronal Network and artificial intelligence for fullyautomatic facial beauty assessment and ranking. A “HOTorNOT” database of1.885 female face images collected from a popular social/dating websitewas used to train the artificial intelligence how to predict facialattractiveness of female faces and rank them.

MESSER U ET AL: “Predicting Social Perception from Faces: A DeepLearning Approach”, ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLINLIBRARY CORNELL UNIVERSITY ITHACA, N.Y. 14853, 29 Jun. 2019(2019-06-29), XP081385771 discloses a Deep Convolutional NeuronalNetwork and artificial intelligence to predict human perceiver'simpression of the characteristics “warmth” and “competence” based on avisual representation of a face of a person. The artificial intelligenceextracts features or face property data from regions of face images andwas trained with a 10K Adults Face database where human raters rated thecharacteristics “warmth” and “competence” of these faces to generateface characteristics. Heat maps were used to identify regions of facesrelevant for human perceivers.

WO 2015/017687A2 discloses a method and system that enables a user totake a picture of their face with a device and send it to a computerserver. The server uses a facial recognition software to identifyanatomical features of the face and the user selects the anatomical areaof his face he wants to improve. In a next step, the server determinesthe level of “severity” of a defect in the selected anatomical area.After that, the person has to select a medical or cosmetic treatmentfrom a list of possible treatments and the server determines the likelytreatment outcome based on data of clinical studies for this selectedmedical or cosmetic treatment. Finally, the server generates a modifiedpicture of the user's face with the likely outcome of the selectedmedical or cosmetic treatment for the selected anatomical area, whichpicture is displayed at the user's device next to the original picturetaken. The disclosed method provides the disadvantage that a user isleft alone both with the selection of facial regions and with theselection of possible medical or cosmetic treatments which might lead toan overall less attractive appearance of the user after one or moretreatments. There is a need for technical means to solve these technicalselection problems.

SUMMARY

These problems are solved with a method that comprises the followingsteps:

-   -   Generate a data set of visuals of faces and extracted face        property data thereof linked to face characteristics data        provided by a representative set of humans that rate the visuals        of these faces about their face characteristics and store the        data set in a database;    -   Extract further face property data of these visuals of faces and        use these extracted face property data together with the        generated data set for training of an artificial intelligence to        enable the artificial intelligence to provide an automated        rating of the characteristics of the visuals of faces;    -   Generate a data set of visual modifications of a face achievable        by cosmetic and/or medical treatments and store the data set in        a database;    -   Take a standardized visual of the face of the person;    -   Input at least one desired characteristic of the face of the        person to be changed;    -   Use the artificial intelligence to analyse the visual of the        person's face and to generate a data set of modifications based        on the selected desired characteristic(s) and modifications        achievable by at least one cosmetic and/or medical treatment;    -   Modify the visual of the face of the person based on the data        set of modifications and generate the computer-modified visual        of the desired face of the person with the modification of the        face achievable by at least one cosmetic and/or medical        treatment;    -   Display the computer-modified visual of the desired face of the        person.

This inventive method, system and/or computer program uses a completedifferent concept and technique to enable an informed decision for auser how to change the facial appearance, expression and/or morphology.The user may be for example, a beautician, dermatologist, physician,specialist certified to do facial modifications or plastic surgeon, aswell as a person interested in a change of their own appearance.

The invention is based on the finding that when aiming to change aperson's appearance it is only a secondary goal to e.g. selectivelyreduce frown lines or to increase the volume of the lips, since personscategorize the appearance of others in the course of forming a firstimpression in a more complex way and as a whole. Characteristicsattributed to a person when making a first impression are, for example,attractiveness, healthiness, youthfulness, tiredness, sadness,friendliness, dominance, competence, likability or trustworthiness justto name some of these. The new concept and technique enables a user toselect at least one characteristic she/he wants to change. A change in acharacteristic attributed by others during forming the first impressioncan be achieved in both ways, i.e. increasing a characteristic which isperceived as positive or decreasing a characteristic which is perceivedas negative. For example, a person might wish to appear less tired. Inanother example, a person might wish to appear more friendly. In afurther example, a person might wish to appear more attractive.

In a first step of the inventive method, a new data set of visuals offaces is compiled. This includes images and videos of people's faces aswell as computer-generated images and 3D models of artificial faces. Ina pre-processing step for the deep learning algorithm, this data set isnormalized via transformations for face alignment, cropping andresizing. Facial landmark detection of face properties first and thenmore detailed like skin texture analysis are performed in order toproperly align face data. The data set of visuals is generated andimproved with human assessments and/or modifications of such faces totrain a deep learning based application and/or artificial intelligenceand/or software. The quality of this application is further refined inan iterative process using data generated by previous versions ofitself. Reference is made to above cited scientific articles as state ofthe art documents that describe the structure and model architecture ofsuch an artificial intelligence.

According to deep learning principles, a new technique is used togenerate a data set of modifications that comprise complex informationabout all modifications of the face needed to change the user'scharacteristics in the desired direction. The modifications might affectthe facial appearance, expression and/or morphology. Such modificationsof the face may comprise, for example, height of the eyebrows, fullnessof the cheeks, width and height of the chin, volume of the lips, volumeof the zygoma, depth of the marionette lines, straightness of the jawline, depth of the glabellar lines, periorbital hollowness and skintightness. Technically, these changes can be obtained by minimallyinvasive and/or invasive cosmetic and/or medical treatments, as has beenproven by clinical studies or treatments performed in the past. Thesetreatments include application of dermal fillers, Botulinum toxin,threads, implants, surgery, laser treatments, autologous fattransplantation, and skin resurfacing treatments amongst others.

In a further step, visual data of the face of the person (e.g. photos orvideos) are obtained. Together with a selection of at least onecharacteristic to be changed, the data set of visuals of faces aretransferred to a server. The additional input of age and/or genderand/or ethnicity data of the person is possible as well.

In a final step of the method, a deep learning basedapplication/artificial intelligence processed by the server is used tomodify the visual data of the user's face according to the user'sselected change of one or more characteristics. The artificialintelligence can also be used to optionally describe and/or rate thecharacteristics based on the original visual data of the person's face.

The computer-modified visualization of the desired face can be shown ona display next to the original visual data of the user to show thepossible change. This change may be obtained by using one or acombination of different invasive and/or minimally invasive cosmeticand/or medical treatments that in sum modify the user's face towards thedesired change of the selected characteristic(s). Optionally, a proposalfor the necessary treatments to achieve the desired face can be given.

This new method provides the major advantage that it visualizes theeffect of changes of the facial appearance, expression and/or morphologyon the characteristics attributed to a person when making a firstimpression. The user has the option to choose at least one specificcharacteristic he/she wishes to change (e.g. reduce or improve).Subsequently, the needed changes of the facial appearance, expressionand/or morphology, which are necessary to reach the desired effect (i.e.the desired face), are visualized.

Methods according to the state of the art visualize only changes ofisolated regions of the face. In the context of characteristicsattributed to a person when making a first impression the face of theuser might be changed in a not desired way. The main aim of abusinessman or politician might be to improve his appearance towardsbeing perceived as a competent person while at the same time he wishesto appear younger. However, the selected changes in the facialcharacteristics to look younger might result in a less competentappearance contradicting his professional needs.

The inventors found that taking into consideration one or morecharacteristics attributed to a person when making a first impression isessential for optimizing the choice and/or selection of individualizeddifferent invasive and/or minimally invasive cosmetic and/or medicaltreatments in order to reach the desired modification of the face. Oneof the requirements for applying this principle is the ability toprocess large data sets in a novel, innovative, fast and efficient wayvia using an artificial intelligence.

Therefore, the invention solves the technical problem of prior art toprocess data in order to analyse, visualize and predict a person's faceand its perception by others according to characteristics attributed tothe person when making a first impression, by a combination of stepslisted above.

BRIEF DESCRIPTION OF THE DRAWINGS

These and further advantageous embodiments of the invention will beexplained based on the following description and the accompanyingdrawings.

FIG. 1 shows a system to display a computer-modified visualization of adesired face of a person.

FIG. 2 shows a mobile device of the system with a picture of the face ofthe person.

FIG. 3 shows in which regions the face of the person is divided forfurther analyses.

FIG. 4 shows a description and rating of the characteristics based onthe original visual data of the person's face.

FIG. 5 shows how characteristics may be selected by the user.

FIG. 6 shows the face of the person with a data set of modificationsoverlaid.

FIG. 7 shows a comparison of the original picture of the face of theperson with the computer-modified visualization of the desired face ofthe person.

FIG. 8 shows a recommendation which treatments to use.

FIG. 9 shows a line drawing of a face with regions of the face marked tobe treated to increase the characteristic “dominant” attributed to aperson when making a first impression.

FIG. 10 shows a picture of a face with regions of the face marked to betreated to increase the characteristic “dominant” attributed to theperson when making a first impression.

FIG. 11 shows a line drawing of a face with regions of the face markedto be treated to increase the characteristic “competence” attributed toa person when making a first impression.

FIG. 12 shows a table with examples of invasive and/or minimallyinvasive cosmetic and medical treatments to achieve changes of desiredcharacteristics of a person's face by actions in particular regions ofthe face of the person.

FIG. 13 shows a visual of a face of a person before and after theperformance of the recommended treatments.

DETAILED DESCRIPTION

FIG. 1 shows a system 1 to display a computer-modified visualization orvisual of a desired face of a person 2 with a mobile device 3. Themobile device 3 processes a software and in particular an App for person2, who considers undergoing an invasive and/or minimally invasivecosmetic and/or medical treatment, or for a specialist performing suchtreatment who would like to take a data-driven decision which treatmentsto choose to obtain the desired changes of the face. A camera of themobile device 3 is used to obtain the visual of the face of person 2 asstandardized visual data 4 shown in FIG. 2. Visual data 4 may representa photo or a film of the face of person 2. Standardization of the visualdata 4 may be split into instructions for person 2 and the photographerwhat to do for taking a standardized photo or film and into apost-processing of the photo or film taken. The instructions for person2 and the photographer may include one or more of the following steps:ask person 2 to take off e.g. earrings or a nose ring; ask person 2 notto smile, ask person 2 to make a neutral facial expression; ask person 2to keep head hair out of his/her face; ask person 2 to look straightinto the camera; good general condition of lightning; neutralbackground. The post-processing of the photo or film may include one ormore of the following steps: cut-out the background behind the face fromthe visual data 4; cut-out the ears of the person's face to reduce thevisual influence of e.g. earrings; cut-out clothes and other wardrobethat might influence with the face; cut-out the head hair of the person2.

System 1 comprises a remote server 5 connected via a broadband network 6or other remote connection technology with the mobile device 3. Theserver 5 processes a deep learning based application 7 and as such formsan artificial intelligence that analyses visual data 4 of the face ofperson 2 to rate one or more characteristics attributed to a person 2when making a first impression. Such face characteristics or traits mayfor example be attractiveness, healthiness, youthfulness, tiredness,sadness, friendliness, dominance, competence, likability ortrustworthiness. The deep learning based applications 7 is a computerprogram comprising instructions which, when the program is executed byremote server 5, causes remote server 5 to carry out the following stepsto provide a computer-modified visualization 13 of a desired face ofperson 2.

To enable the deep learning based application 7 to rate facecharacteristics the following steps are processed. In a first step adata set of visual data of visuals of faces and extracted face propertydata thereof linked to face characteristics data is generated. Toextract face properties conventional computer vision algorithms like alandmark detection divide the face of person 2 in regions like the chinand the jawline as shown in FIG. 3 and automatically extractscharacteristics and their location in the face. Such face property datamay for instance include the distance between the eyes or the distancebetween the eyes and the mouth and other distances to be measure todescribe a face. These face property data are stored together with thevisual data of these faces by the deep learning based application 7 in adatabase 8 of the server 5. A representative number of such visuals offaces stored as visual data in database 8 are shown on a display to arepresentative number of humans to manually rate these visuals of facesabout their characteristics. The humans may rate them with scores (e.g.from 0 to 7) for different characteristics or traits. These humanratings are stored in database 8 linked to the visual data of the facesand provide a basis information for the deep learning based application7 to rate characteristics attributed to a person 2 when making a firstimpression.

In a second step further face property data of these visuals of facesare extracted by conventional computer vision algorithms for examplelandmark detection, wrinkle detection, skin texture analysis, analysisof facial proportions. These face property data of visuals of faces areused together with the data set generated and stored in database 8 inthe first step for training of the deep learning based application 7 toenable the artificial intelligence to provide an automated rating of thecharacteristics of the visuals of faces. Reference is made to thescientific articles listed above that describe the structure and modelarchitecture of such an artificial intelligence. As a result, any visualof a face may be provided to the deep learning based application 7,which will, based on the data set stored in database 8, provide anautomated rating of the characteristics of the visuals of the face. FIG.4 shows the result of such a description and automated rating of thecharacteristics or traits of a person based on the visuals of theperson's face displayed on mobile device 3.

Server 5 furthermore comprises a database 9 with data generated in athird step based on clinical studies, case studies or other publiclyavailable information, which data comprise information about visualmodifications of a face achievable by invasive and/or minimally invasivecosmetic and medical treatments. This database 9 for instance comprisesinformation of the effectiveness of a treatment improving a wrinklescore of 3.1 to 1.5 within 3 weeks. Reference is made to prior art WO2015/017687 that discloses parameters-based clinical trial summaries,for example the investigator's rating of glabellar line severity atmaximum frown and the subject's global assessment of change inappearance of glabellar lines after a medical treatment with Botox overa time period of 120 days post-injection. This prior art documentdiscloses a database that stores data how to improve one particular areaof a face with one particular medical treatment. FIG. 12 as one furtherexample shows a table, which includes the information of database 9 withexamples of invasive and/or minimally invasive cosmetic and medicaltreatments to achieve actions or improvements in particular regions ofthe face of person 2.

After databases 8 and 9 have been setup with above described threesteps, system 1 is ready to be used to provide computer-modified visualsof a face of a person as described in the following steps of the method.

In a fourth step the camera of mobile device 3 is used to obtain thestandardized visual data 4 of the face of person 2 as described aboveand shown in FIG. 2. In a preferred optional embodiment, these visualdata 4 are sent to sever 5 and deep learning based application 7processes an automated rating of the characteristics or traits of person2 and provides the rating shown in FIG. 4 to support person 2 to decidewhich characteristic or trait he/she might want to change. In anotherless preferred embodiment person 2 makes his/her decision whichcharacteristic of his/her face to improve without the automated ratingshown in FIG. 4.

In a fifth step person 2 inputs at least one characteristic of the faceof the person to be changed based on the personal interest of person 2.The characteristic(s) input by person 2 is stored and transmitted in theform of face characteristics data. The person 2 may decide to changehis/her facial appearance, expression and/or morphology for improvingthe characteristic “competent” with input means 10 of mobile device 3 asshown in FIG. 5. This characteristic selected by person 2 is transmittedvia broadband network 6 to server 5. In another embodiment of theinvention person 2 may use another way to input the at least onecharacteristic of the face he/she is interested to change with mobilephone 3. This may be done by the selection of the App used by person 2,as there may be an App to improve the characteristic “competent” andanother App to improve the characteristic “dominant”.

In a sixth step of the method the artificial intelligence of the server5 analyses the visual of the person's face and generates a data set ofmodifications 12 based on the selected desired characteristic(s) andmodifications achievable by at least one cosmetic and/or medicaltreatment of database 9. This means that either only one or morecosmetic treatments or only one or more medical treatments or anycombination of one or more cosmetic and medical treatments of database 9may be used to modify the user's face towards the desired change of theselected desired characteristic(s). Furthermore, references to acosmetic treatment are meant to cover any invasive and/or minimallyinvasive cosmetic treatment. To achieve this technical step, the deeplearning based application 7, based on database 8, evaluates whichmodifications are needed to improve the characteristic “competent” ofperson 2 and matches these modifications needed with modificationspossible as stored in database 9. FIG. 11 shows a line drawing of a facewith regions of the face marked to be treated to increase thecharacteristic “competent” attributed to a person when making a firstimpression. In line with that, the table of FIG. 12 shows the actionsneeded to increase the characteristic “competent” attributed to person 2when making a first impression: make the chin less wide and the cheeksless full and lower the eyebrows. FIG. 12 furthermore shows the cosmeticand/or medical treatments, which can be performed by a beautician,dermatologist, physician, specialist certified to do facialmodifications, or plastic surgeon to realize these actions.

The result of a best match of modifications needed and modificationspossible are then stored in the data set of modifications 12. This dataset of modifications 12 technically describes what modifications areneeded to modify visual data 4 of the face of person 2 to show thepossible result of one or more invasive and/or minimally invasivecosmetic and/or medical treatments to improve the characteristic“competent” of person 2 in a computer-modified visual 13 of the face ofa person 2 as shown in FIG. 6. FIG. 6 shows the face of person 2 with anoverlay of arrows that indicate with regions of the face need to betreated to achieve the desired result of improved “competence”. So forinstance, the eyebrow position needs to be lifted and the volume of thejawline needs to be increased. The arrows shown are only symbolic asdata set of modifications 12 may comprise further information about theface and processing of the visual data 4 needed.

In as seventh step of the method the visual data 4 of the face of theperson 2 are modified based on the data set of modifications 12 and acomputer-modified visual 13 of the face of the person 2 with themodification of the face achievable by the at least one proposedcosmetic and/or medical treatment is generated. If for instanceartificial intelligence of the server 5 concluded that the eye bags ofperson 2 need to be tightened to improve the characteristic“attractiveness” of person 2, then the data set of modifications 12 mayinclude information to soft focus the area of lower eyelid and zygoma inthe visual data 4 of the face of person 2. Deep learning basedapplication 7 or other conventional image processing methods like imagewarping therefore processes the photo or film of person 2 to provide thecomputer-modified visual 13 of the desired face of person 2. Thisseventh step may be processed by server 5 due to its large processingpower, but could be processed by mobile device 3 as well. In thisembodiment server 5 therefore comprises picture data modification means14 to modify the visual data 4 of the face of person 2 according to thedata set of modifications 12, which modified visual data 4 of the faceof person 2 are transmitted to mobile device 3 and displayed ascomputer-modified visual 13 of the desired face of person 2 with mobiledevice 3.

In a preferred embodiment of the invention the artificial intelligenceis used to automatically identify an area with wrinkles in the visual ofthe person's face based on technologies known to a man skilled in theart. Such areas for instance may be the area of lower eyelid and zygoma.The artificial intelligence may then be used to automatically soft focusthese identified areas in case for instance the characteristic“attractiveness” of person 2 should be improved and the data set ofmodifications therefore includes such information to modify the visualof the face of the person to generate the computer-modified visual ofthe desired face of the person. In a further preferred embodiment, theartificial intelligence may also add wrinkles to the visual of a forinstance young person's face, who wants to improve the characteristic“competence” in areas where elder people use to have wrinkles.

In an eighth step of the method the computer-modified visual 13 of thedesired face of the person 2 is displayed with mobile device 3 as shownin FIG. 7. There are two preferred modes to display the visual data 4 ofthe face and the computer-modified visual 13 of the desired face ofperson 2 to enable him/her to easier see the differences. The firstpreferred mode is to use a toggle mode to alternatively show the takenstandardized visual data 4 of the face of the person 2 and thecomputer-modified visual 13 of the desired face of the person 2. Person2 just has to touch the display of mobile device 3 in a button area totoggle between the two visuals as fast as person 2 wants to see them tobetter see the differences and modifications. The second preferred modeis to use a marking mode to mark the areas of the face of the person 2modified by the data set of modifications 12 as an overlay to thedisplayed computer-modified visual 13 of the desired face of the person2 as shown in FIG. 10. Marking may be done e.g. with lines or brokenlines overlaid over the computer-modified visual 13 of the desired faceof the person 2. Both preferred modes enable person 2 to easily seethose areas of the face that would need to be treated with invasiveand/or minimally invasive cosmetic and/or medical treatments.

FIG. 13 shows the face of another person 2 before and after therecommended treatment was performed. The left photo shows person 2 whowas interested to change the appearance of his face by increasing thecharacteristics “dominance” and “competence”. The inventive method andsystem provided and displayed a computer-modified visual 13 of thedesired face similar to the right photo and provided a recommendation touse the treatment of lipofilling in particular identified regions of theface. After the recommended treatment was performed, the right photo ofFIG. 13 was taken and it turned out that the computer-modified visual 13of the desired face was nearly identical to the actual photo taken afterthe treatment. This technical solution helps substantially to makeinformed decisions about cosmetic and/or medical treatments.

It is furthermore possible to display all invasive and/or minimallyinvasive cosmetic and/or medical treatments stored in database 9 and toselect some of these invasive and/or minimally invasive cosmetic and/ormedical treatments upfront to send these together with the visual data 4of person 2 and the characteristic input by person 2 to change his/herfacial appearance to server 5. In this case, the artificial intelligenceof server 5 only uses those selected invasive and/or minimally invasivecosmetic and/or medical treatments during their search for a best matchachievable for the data set of modifications 12 in the database 9 ofvisual modifications achievable by selected invasive and/or minimallyinvasive cosmetic and/or medical treatments. This enables person 2 todecide upfront which of the invasive and/or minimally invasive cosmeticand/or medical treatments are acceptable to be used to change his/herfacial appearance and helps to streamline processing of server 5.

In case the sevenths step of above described method is processed bymobile device 3 and not on server 5, then mobile device 3 would comprisevisual data modification means 14 to modify the visual data 4 of theface of the person 2 with the received data set of modifications 12 fromserver 5 by altering certain image vectors. The computer-modified visual13 of the desired face of the person 2 is then displayed with the device3. In another embodiment of the invention image processing by visualdata modification means might be split between the server 5 and mobiledevice 3.

Deep learning based application 7 is optionally built to evaluate theage period, ethnicity and gender of person 2 in picture data 4. Thishelps to reduce data needed to be input when using the App.

System 1 furthermore enables to show those invasive and/or minimallyinvasive cosmetic and/or medical treatments 16 on the display of mobiledevice 3 that have been selected by the artificial intelligence toachieve the desired face as shown in FIG. 8. Person 2 may decide to usea filter to select only some of the shown invasive and/or minimallyinvasive cosmetic and/or medical treatments, if for instance person 2 isnot willing to undergo a surgical intervention. This selection of person2 is sent to server 5, which calculates the necessary data set ofmodifications 12, which are achievable with the reduced number ofinvasive and/or minimally invasive cosmetic and/or medical treatments toachieve the desired changes in the characteristic (e.g. attractiveness).

It is furthermore possible to input more than one characteristic withinput means of mobile device 3 as shown in FIG. 5. This provides theadvantage that person 2 is free to improve two or more characteristicsbased on his/her personal interest.

It is furthermore possible that the person that uses the App is not theperson that wants to change his/her appearance, but a person that wantsto enable an informed decision like for example, a beautician,dermatologist, physician, specialist certified to do facialmodifications or plastic surgeon.

In a further embodiment of the invention, the computer program realizedas an APP of a mobile phone may be programmed to ask the user questionslike the following: gender, age, profession, level of education, sexualorientation, religion and political orientation. These questions may beasked in the fifth step of above explained method about the user in oneembodiment and about the target group, the user is interested in, in asecond embodiment. This information may be used in the sixth step ofabove explained method, when generating the data set of modifications.The result of an analyse of the information about the user and/or thetarget group, the user is interested in, for which the user wants to berecognized as e.g. more “dominant” may be used as further input howcharacteristics of the user need to be modified. This has the advantagethat the modifications closely fit the personal needs and wishes of theuser.

In a further embodiment of the invention, an artificial intelligencesystem is used to provide an automated rating of the characteristic ofthe visuals of faces. Such artificial intelligence system may include asoftware to detect landmarks in the face of a person or any otherconventional algorithm with the ability to do so.

The invention claimed is:
 1. A method for providing a computer-modifiedvisualization of a desired face of a person, comprising the steps of:generating a first data set of visuals of faces and extracted faceproperty data thereof linked to face characteristics data provided by arepresentative set of humans and storing the first data set, wherein thefirst data set is utilized to rate face characteristics of the visualsof faces; extracting further face property data of the visuals of faces;training an artificial intelligence to enable the artificialintelligence to provide an automated rating of the face characteristicsof the visuals of faces using the extracted further face property datatogether with the generated first data set; generating a second data setof visual modifications of a particular face achievable by a cosmeticand/or a medical treatment and storing the second data set; taking astandardized visual of a particular face of a person; inputting at leastone desired characteristic of the particular face of the person to bechanged; using the artificial intelligence to analyse a particularvisual of the particular face of the person and to generate a third dataset of at least one desired visual modification of the person based onthe at least one desired characteristic and at least one visualmodification achievable by the cosmetic and/or the medical treatment;modifying the standardized visual of the particular face of the personbased on the third data set of the at least one desired visualmodification; generating the computer-modified visual of a desired faceof the person with the at least one visual modification of the faceachievable by the cosmetic and/or the medical treatment; and displayingthe computer-modified visual of the desired face of the person.
 2. Themethod according to claim 1, further comprising the additional steps of:using a toggle mode to alternatively show the standardized visual of theparticular face of the person and the computer-modified visual of thedesired face of the person, or using a marking mode to mark areas of thedisplayed computer-modified visual of the desired face of the personmodified by the third data set of the desired visual modifications as anoverlay to the displayed computer-modified visual of the desired face ofthe person.
 3. The method according to claim 1, further comprising thesteps of: displaying all of the cosmetic and/or the medical treatmentsstored in the second data set; selecting only one or more of thecosmetic and/or the medical treatments displayed; and using only theselected cosmetic and/or the medical treatments for the generation ofthe third data set.
 4. The method according to claim 1, furthercomprising the additional steps of: displaying the face characteristics;and selecting two or more of the face characteristics displayed as theat least one desired characteristic to be used by the artificialintelligence to generate the third data set of visual modifications. 5.The method according to claim 1, further comprising the additional stepof: displaying a particular cosmetic and/or a particular medicaltreatment selected by the artificial intelligence to be used to generatethe third data set of the at least one desired visual modification. 6.The method according to claim 1, further comprising the additional stepsof: using the artificial intelligence to analyse the particular visualof the particular face of the person to provide an automated rating of aparticular face characteristic of the particular visual of theparticular face of the person; and displaying the particular facecharacteristics of the particular visual of the particular face of theperson.
 7. The method according to claim 1, further comprising theadditional steps of: using the artificial intelligence to automaticallyidentify an area with wrinkles in the visual of the particular visual ofthe particular face of the person and to automatically soft focus theidentified area.
 8. A system for displaying a computer-modifiedvisualization of a desired face of a person with a mobile device,wherein the mobile device comprises a display and a camera configured toinput a face characteristic, and the mobile device is connected to aremote server which generates a data set of visual modifications,wherein: the remote server comprises a deep learning based applicationto process the steps of the method according to claim 1 to generate afirst database with data sets of visuals of faces and extracted faceproperty data thereof linked to face characteristics and to generate asecond database with the data set of visual modifications of a faceachievable by cosmetic and/or medical treatments and the deep learningbased application is built to generate the data set of visualmodifications for at least one region of the face of the person toachieve a desired change in a particular face characteristic based onthe generated databases; and the remote server or the mobile deviceconfigured to modify visual data of the face of the person with the dataset of visual modifications achievable by at least one cosmetic and/ormedical treatment, wherein the modified visual data of the face aredisplayed with the mobile device.
 9. A non-transitory computer readablemedium storing a computer program comprising instructions which, whenthe program is executed by a computer, causes the computer to carry outthe following steps to provide a computer-modified visualization of adesired face of a person: generating a first data set of visuals offaces and extracted face property data thereof linked to facecharacteristics data provided by a representative set of humans andstoring the first data set, wherein the first data set is utilized torate face characteristics of the visuals of faces; extracting furtherface property data of the visuals of faces; training an artificialintelligence to enable the artificial intelligence to provide anautomated rating of the face characteristics of the visuals of facesusing the extracted further face property data together with thegenerated first data set; generating a second data set of visualmodifications of a particular face achievable by cosmetic and/or medicaltreatments and storing the second data set; receiving a standardizedvisual of a particular face of a person from a mobile device; receiving,from the mobile device, at least one desired characteristic of theparticular face of the person to be changed; using the artificialintelligence to analyse a particular visual of the particular face ofthe person and to generate a third data set of a desired visualmodification of the person based on the inputted at least one desiredcharacteristic and at least one of the visual modifications achievableby the cosmetic and/or the medical treatment; modifying the standardizedvisual of the particular face of the person based on the third data setof the at least one desired visual modification; generating thecomputer-modified visual of a desired face of the person with the atleast one visual modification of the face achievable by the cosmeticand/or the medical treatment; and transmitting the computer-modifiedvisual of the desired face of the person to the mobile device to displaythe computer-modified visual.
 10. The non-transitory computer readablemedium according to claim 9, the computer program further comprisinginstructions which, when the program is executed by a computer, causethe computer to carry out the following additional steps of: using atoggle mode to alternatively show the standardized visual of theparticular face of the person and the computer-modified visual of thedesired face of the person, or using a marking mode to mark areas of thedisplayed computer-modified visual of the desired face of the personmodified by the third data set of the desired visual modifications as anoverlay to the displayed computer-modified visual of the desired face ofthe person.
 11. The non-transitory computer readable medium according toclaim 9, the computer program further comprising instructions which,when the program is executed by a computer, cause the computer to carryout the following additional steps of: displaying all of the cosmeticand/or the medical treatments stored in the second data set; selectingonly one or more of the cosmetic and/or medical treatments displayed;and using only the selected cosmetic and/or the medical treatments forthe generation of the third data set of.
 12. The non-transitory computerreadable medium according to claim 9, the computer program furthercomprising instructions which, when the program is executed by acomputer, cause the computer to carry out the following additional stepsof: displaying the face characteristics; and selecting two or more ofthe face characteristics displayed as the at least one desiredcharacteristic to be used by the artificial intelligence to generate thethird data set of visual modifications.
 13. The non-transitory computerreadable medium according to claim 9, the computer program furthercomprising instructions which, when the program is executed by acomputer, cause the computer to carry out the following additional stepsof: displaying all characteristics stored in the database; and selectingtwo or more of the characteristics displayed as desired characteristicto be used by the artificial intelligence to generate the data set ofmodifications.
 14. The non-transitory computer readable medium accordingto claim 9, the computer program further comprising instructions which,when the program is executed by a computer, cause the computer to carryout the following additional step of: displaying a particular cosmeticand/or a particular medical treatment selected by the artificialintelligence to be used to generate the third data set of at least onedesired visual modification.
 15. The non-transitory computer readablemedium according to claim 9, the computer program further comprisinginstructions which, when the program is executed by a computer, causethe computer to carry out the following additional steps of: using theartificial intelligence to analyse the particular visual of theparticular face of the person to provide an automated rating of aparticular face characteristic of the particular visual of theparticular face of the person; and displaying the particular facecharacteristics of the particular visual of the particular face of theperson.
 16. The non-transitory computer readable medium according toclaim 9, the computer program further comprising instructions which,when the program is executed by a computer, cause the computer to carryout the following additional steps of: using the artificial intelligenceto automatically identify an area with wrinkles in the visual of theparticular visual of the particular face of the person and toautomatically soft focus the identified area.
 17. A system for providinga modified visual of a face of a person, comprising: a remote serverconfigured to: generate a first data set of visuals of faces andextracted face property data thereof linked to face characteristics dataprovided by a representative set of humans and store the first data set,wherein the first data set is utilized to rate face characteristics ofthe visuals of faces; extract further face property data of the visualsof faces; train an artificial intelligence to enable the artificialintelligence to provide an automated rating of the face characteristicsof the visuals of faces using the extracted further face property datatogether with the generated first data set; generate a second data setof visual modifications of a particular face achievable by cosmeticand/or medical treatments and storing the second data set; receive astandardized visual of a particular face of a person; receive at leastone desired characteristic of the particular face of the person to bechanged; use the artificial intelligence to analyse a particular visualof the particular face of the person and to generate a third data set ofat least one desired visual modification of the person based on the atleast one desired characteristic and at least one visual modificationachievable by the cosmetic and/or the medical treatment; modify thestandardized visual of the particular face of the person based on thethird data set of the at least one desired visual modification; generatea computer-modified visual of a desired face of the person with the atleast one visual modification of the face achievable by the cosmeticand/or the medical treatment; and display the computer-modified visualof the desired face of the person.
 18. The system according to claim 17,the remote server further configured to: use a toggle mode toalternatively show the standardized visual of the particular face of theperson and the computer-modified visual of the desired face of theperson, or use a marking mode to mark areas of the displayedcomputer-modified visual of the desired face of the person modified bythe third data set of the desired visual modifications as an overlay tothe displayed computer-modified visual of the desired face of theperson.
 19. The system according to claim 17, the remote server furtherconfigured to: display all of the cosmetic and/or the medical treatmentsstored in the second data set; select only one or more of the cosmeticand/or medical treatments displayed; and use only the selected cosmeticand/or the medical treatments for the generation of the third data setof.
 20. The system according to claim 17, the remote server furtherconfigured to: display the face characteristics; and select two or moreof the face characteristics displayed as the at least one desiredcharacteristic to be used by the artificial intelligence to generate thethird data set of visual modifications.
 21. The system according toclaim 17, the remote server further configured to: display allcharacteristics stored in the database; and select two or more of thecharacteristics displayed as desired characteristic to be used by theartificial intelligence to generate the data set of modifications. 22.The system according to claim 17, the remote server further configuredto: display a particular cosmetic and/or a particular medical treatmentselected by the artificial intelligence to be used to generate the thirddata set of at least one desired visual modification.
 23. The systemaccording to claim 17, the remote server further configured to: use theartificial intelligence to analyse the particular visual of theparticular face of the person to provide an automated rating of aparticular face characteristic of the particular visual of theparticular face of the person; and display the particular facecharacteristics of the particular visual of the particular face of theperson.
 24. The system according to claim 17, the remote server furtherconfigured to: use the artificial intelligence to automatically identifyan area with wrinkles in the visual of the particular visual of theparticular face of the person and to automatically soft focus theidentified area.